-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add moz-webgpu-cts process-reports
subcmd. with multiple --preset
s
#14
Conversation
cd7902c
to
454a4db
Compare
I was struggling to get traction on resolving conflicts between expected outcomes in metadata and reported outcomes. I realized I was missing an answer to the question, “How do I structure new expectations' conditional values once reports have been analyzed?” An example answer: one could organize by platform: [test]
[subtest]
expected:
if os == "win": FAIL
if os == "linux": TIMEOUT …or one could organize by build profile: [test]
[subtest]
expected:
if debug: FAIL
if not debug: TIMEOUT …or even potentially mix them, which seems reasonable, but only muddies up the solution space for “what do we emit”: [test]
[subtest]
expected:
if os == "win": NOTRUN
if debug: FAIL In addition to the variables explored above, WPT upstream lets users define what I'll call “branching points” in expectation structure for any run info variable. WPT upstream's I determined to answer the question I've posed instead by (1) baking a branch point configuration into code for now, and (2) granting Footnotes
|
4d39480
to
eb6f649
Compare
20f8b91
to
9a92a59
Compare
cfa3114
to
d35ca95
Compare
moz-webgpu-cts adjust-exps
subcmd.moz-webgpu-cts process-reports
subcmd.
d35ca95
to
2cedac7
Compare
323f8a3
to
02ea3ee
Compare
09caea1
to
3705a3b
Compare
Co-authored-by: Jim Blandy <[email protected]>
3705a3b
to
fe5ada9
Compare
moz-webgpu-cts process-reports
subcmd.moz-webgpu-cts process-reports
subcmd. with multiple --preset
s
Resolves #13, #9.
Depends on:
whippit
to be a set oftraits
instead ofstruct
s #40